当样本通过深层神经网络时，功能，逻辑和标签是三个主要数据。近年来，功能扰动和标签扰动受到越来越多的关注。事实证明，它们在各种深度学习方法中很有用。例如，（对抗性）特征扰动可以提高学习模型的鲁棒性甚至概括能力。但是，有限的研究已明确探索了对逻辑向量的扰动。这项工作讨论了几种与类级别logit扰动有关的现有方法。建立了logit扰动引起的正/负数据扩大和损失变化之间的统一观点。提供理论分析以阐明为什么类级logit扰动有用。因此，提出了新的方法，以明确学习单标签和多标签分类任务的扰动逻辑。基准图像分类数据集及其长尾版本的广泛实验表明我们的学习方法的竞争性能。由于它仅在logit上，因此可以用作与任何现有分类算法融合的插件。所有代码均可在https://github.com/limengyang1992/lpl上找到。

最近，机器学习（ML）电位的发展使得以量子力学（QM）模型的精度进行大规模和长期分子模拟成为可能。但是，对于高水平的QM方法，例如在元gga级和/或具有精确交换的密度函数理论（DFT），量子蒙特卡洛等，生成足够数量的用于训练的数据由于其高成本，计算挑战性。在这项工作中，我们证明了基于ML的DFT模型Deep Kohn-Sham（Deepks）可以在很大程度上缓解这个问题。 DeepKS采用计算高效的基于神经网络的功能模型来构建在廉价DFT模型上添加的校正项。在训练后，DeepKs提供了与高级QM方法相比，具有紧密匹配的能量和力，但是所需的训练数据的数量是比训练可靠的ML潜力所需的数量级要小。因此，DeepKs可以用作昂贵的QM型号和ML电位之间的桥梁：一个人可以生成相当数量的高准确性QM数据来训练DeepKs模型，然后使用DeepKs型号来标记大量的配置以标记训练ML潜力。该周期系统方案在DFT软件包算盘中实施，该计划是开源的，可以在各种应用程序中使用。

由于学习难度对于机器学习至关重要（例如，基于难度的加权学习策略），以前的文献提出了许多学习难度措施。但是，迄今为止尚无针对学习难度的全面调查，导致几乎所有现有的措施都在没有严格的理论基础的情况下进行了启发性定义。此外，即使在许多研究中至关重要，也没有正式的简单和硬样品定义。这项研究试图进行一项试验理论研究，以实现样本的学习难度。首先，根据概述误差的偏见变化权衡理论提出了学习难度的理论定义。基于拟议的定义建立了简单和硬样品的理论定义。从正式定义中给出了一种实用的学习难度测量方法。其次，探索了学习难度的加权策略的属性。随后，可以根据探索的属性来很好地解释机器学习中的几种经典加权方法。第三，评估提出的措施以验证其合理性和优越性，以几个主要的难度因素。这些实验中的比较表明，所提出的措施在整个实验过程中的其他措施显着优于其他措施。

Color fundus photography and Optical Coherence Tomography (OCT) are the two most cost-effective tools for glaucoma screening. Both two modalities of images have prominent biomarkers to indicate glaucoma suspected. Clinically, it is often recommended to take both of the screenings for a more accurate and reliable diagnosis. However, although numerous algorithms are proposed based on fundus images or OCT volumes in computer-aided diagnosis, there are still few methods leveraging both of the modalities for the glaucoma assessment. Inspired by the success of Retinal Fundus Glaucoma Challenge (REFUGE) we held previously, we set up the Glaucoma grAding from Multi-Modality imAges (GAMMA) Challenge to encourage the development of fundus \& OCT-based glaucoma grading. The primary task of the challenge is to grade glaucoma from both the 2D fundus images and 3D OCT scanning volumes. As part of GAMMA, we have publicly released a glaucoma annotated dataset with both 2D fundus color photography and 3D OCT volumes, which is the first multi-modality dataset for glaucoma grading. In addition, an evaluation framework is also established to evaluate the performance of the submitted methods. During the challenge, 1272 results were submitted, and finally, top-10 teams were selected to the final stage. We analysis their results and summarize their methods in the paper. Since all these teams submitted their source code in the challenge, a detailed ablation study is also conducted to verify the effectiveness of the particular modules proposed. We find many of the proposed techniques are practical for the clinical diagnosis of glaucoma. As the first in-depth study of fundus \& OCT multi-modality glaucoma grading, we believe the GAMMA Challenge will be an essential starting point for future research.

用于培训样本的有效加权方案对于学习任务至关重要。已经提出了许多加权方案。有些方案采用易于第一模式，而其他一些则采取艰难的第一模式。自然而然，提出了一个有趣但实际的问题。应该首先学习哪些样本，而且很容易或努力？为了回答这个问题，进行理论分析和实验验证。首先，提出了一般优化的目标函数，揭示了难度分布与基于困难的样本权重之间的关系。其次，在优化的目标函数的基础上，获得理论答案。除了易于第一和良好的第一模式之外，还有另外两种优先模式，即中等第一和两端 - 首先。在培训过程中，先前模式不一定保持不变。第三，建议在没有先前的知识或理论线索时选择有效和通用的解决方案以选择最佳优先模式。四种模式，即易于/中/硬/二端 - 首先，可以在所提出的解决方案中灵活地切换。第四，在各种场景下进行广泛的实验，以进一步比较不同模式的加权方案。在这些作品的基础上，获得合理和全面的答案。包括样本学习困难分布的因素和验证数据确定应该首先在学习任务中学习哪些样本。

机器学习中的加权战略占上风。例如，强大的机器学习中的常见方法是对可能是噪声或非常难以造成的样本的较低重量。本研究揭示了另一种未被发现的策略，即补偿。已经利用了各种补偿的化身，但尚未明确揭示。学习赔偿称为补偿学习，并在本研究中为其构建系统分类。在我们的分类学中，赔偿学习是根据赔偿目标，方向，推理方式和粒度水平分开的。可以至少部分地视为补偿技术，包括一些现有的学习算法包括一些经典的学习算法。此外，可以通过将补偿学习插入现有的学习算法来获得一系列新的学习算法。具体而言，提出了两种混凝土新的学习算法，用于强大的机器学习。关于图像分类和文本情绪分析的广泛实验验证了两种新算法的有效性。补偿学习也可以用于其他各种学习场景，例如不平衡学习，聚类，回归等。

机器学习中的常见假设是样本独立地和相同地分布（i.i.d）。然而，不同样本的贡献在训练中并不相同。一些样品难以学习，一些样品是嘈杂的。样品的不等贡献对培训表演具有相当大的影响。在学习方面，专注于不平等的样本贡献（例如，容易，艰难，嘈杂）的研究通常将这些贡献称为强大的机器学习（RML）。称重和正规化是RML中的两种常见技术。已经提出了许多学习算法，但处理容易/硬/噪声样本的策略不同或甚至与不同的学习算法相矛盾。例如，一些策略首先采取硬样品，而某些策略首先则轻松。由于RML缺乏统一的理论框架，对处理不同样本的现有RML算法进行明确的比较。本研究试图基于偏差差异折衷理论来构建RML的数学基础。提出并证明了一系列定义和属性。还解释并比较了几种古典学习算法。基于比较获得现有方法的改进。提出了一种结合两个古典学习策略的统一方法。

This paper illustrates the technologies of user next intent prediction with a concept knowledge graph. The system has been deployed on the Web at Alipay, serving more than 100 million daily active users. Specifically, we propose AlipayKG to explicitly characterize user intent, which is an offline concept knowledge graph in the Life-Service domain modeling the historical behaviors of users, the rich content interacted by users and the relations between them. We further introduce a Transformer-based model which integrates expert rules from the knowledge graph to infer the online user's next intent. Experimental results demonstrate that the proposed system can effectively enhance the performance of the downstream tasks while retaining explainability.

Deep neural networks (DNNs) are found to be vulnerable to adversarial attacks, and various methods have been proposed for the defense. Among these methods, adversarial training has been drawing increasing attention because of its simplicity and effectiveness. However, the performance of the adversarial training is greatly limited by the architectures of target DNNs, which often makes the resulting DNNs with poor accuracy and unsatisfactory robustness. To address this problem, we propose DSARA to automatically search for the neural architectures that are accurate and robust after adversarial training. In particular, we design a novel cell-based search space specially for adversarial training, which improves the accuracy and the robustness upper bound of the searched architectures by carefully designing the placement of the cells and the proportional relationship of the filter numbers. Then we propose a two-stage search strategy to search for both accurate and robust neural architectures. At the first stage, the architecture parameters are optimized to minimize the adversarial loss, which makes full use of the effectiveness of the adversarial training in enhancing the robustness. At the second stage, the architecture parameters are optimized to minimize both the natural loss and the adversarial loss utilizing the proposed multi-objective adversarial training method, so that the searched neural architectures are both accurate and robust. We evaluate the proposed algorithm under natural data and various adversarial attacks, which reveals the superiority of the proposed method in terms of both accurate and robust architectures. We also conclude that accurate and robust neural architectures tend to deploy very different structures near the input and the output, which has great practical significance on both hand-crafting and automatically designing of accurate and robust neural architectures.

A computational graph in a deep neural network (DNN) denotes a specific data flow diagram (DFD) composed of many tensors and operators. Existing toolkits for visualizing computational graphs are not applicable when the structure is highly complicated and large-scale (e.g., BERT [1]). To address this problem, we propose leveraging a suite of visual simplification techniques, including a cycle-removing method, a module-based edge-pruning algorithm, and an isomorphic subgraph stacking strategy. We design and implement an interactive visualization system that is suitable for computational graphs with up to 10 thousand elements. Experimental results and usage scenarios demonstrate that our tool reduces 60% elements on average and hence enhances the performance for recognizing and diagnosing DNN models. Our contributions are integrated into an open-source DNN visualization toolkit, namely, MindInsight [2].